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APPARATUS AND METHOD FOR available to users, substantial interest has developed in the 

OPTIMIZING THE PERFORMANCE OF area of intelligent agent computer programs, also referred to 

COMPUTER TASKS USING MULTIPLE as intelligent agents or simply agents, that operate much like 

INTELLIGENT AGENTS HAVING VARIED software-implemented "assistants" to automate and simplify 

DEGREES OF DOMAIN KNOWLEDGE S certain tasks in a way that hides their complexity from the 

user. With agents, a user may be able to perform tasks 

CROSS-REFERENCE TO RELATED without having to know specific sequences of commands. 

APPLICATIONS Similarly, a user may be able to obtain information without 

This application is related to the following U.S. Patent havin S to kn° w exactly how or where to search for the 

Applications, all of which were filed on even date herewith 10 information. 

by Bigus et al: U.S. Ser. No. 08/821,935 entitled "INTEL- Intelligent agents are characterized by the concept of 

LIGENT AGENT WITH NEGOTIATION CAPABILITY delegation, where a user, or client, entrusts the agents to 

AND METHOD OF NEGOTIATION THEREWITH", U.S. handle tasks with at least a certain degree of autonomy 

Ser. No. 08/822,119 entitled "APPARATUS AND Intelligent agents operate with varying degrees of con- 

METHOD FOR COMMUNICATING BETWEEN AN 15 straints depending upon the amount of autonomy that is 

INTELLIGENT AGENT AND CLIENT COMPUTER delegated to them by the user. 

PROCESS USING DISGUISED MESSAGES" and U.S. Intelligent agents may also have differing capabilities in 

Ser. No. 08/822,993 entitled "APPARATUS AND terms of intelligence, mobility, agency, and user interface. 

METHOD FOR OPTIMIZING THE PERFORMANCE OF Intelligence is generally the amount of reasoning and deci- 

COMPUTER TASKS USING INTELLIGENT AGENT 20 sion making that an agent possesses. This intelligence can be 

WITH MULTIPLE PROGRAM MODULES HAVING aS simple as following a predefined set of rules, or as 

VARIED DEGREES OF DOMAIN KNOWLEDGE". The complex as learning and adapting based upon a user's 

disclosures of ail of these applications are hereby incorpo- objectives and the agent's available resources. An intelligent 

rated by reference herein. agent's intelligence or skill as applied to a specific field or 

FIELD OF THE INVENTION * hB «™} S K& ™ d J° aS d ° main 

Mobility is the ability to be passed through a network and 

The invention is generally related to intelligent agent execute on different computer systems. That is, some agents 

computer programs executable on computer systems and the m ay be designed to stay on one computer system and may 

like, and in particular, the use of such agents for performing ^ ne ver be passed to different machines, while other agents 

specific tasks in a computer system. may be mobile in the sense that they are designed to be 

BACKGROUND OF THF INVFNTtON p assed from computer to computer whUe performing tasks at 

a ALMjKU un V l ii£ iin vun 1 1UIN different stops along the way. User interface defines how an 

Since the advent of the first electronic computers in the agent interacts with a user, if at all. 

1940's, computers have continued to handle a greater vari- 35 Agents have a number of uses in a wide variety of 

ety of increasingly complex tasks. Advances in semicon- applications, including systems and network management, 

ductors and other hardware components have evolved to the mobile access and management, information access and 

point that current low-end desktop computers can now management, collaboration, messaging, workflow and 

handle tasks that once required roomfuls of computers. administrative management, and adaptive user interfaces. 

Computer programs, which are essentially the sets of 40 Another important use for agents is in electronic commerce, 
instructions that control the operation of a computer to where an agent may be configured to seek out other parties 
perform tasks, have also grown increasingly complex and such as other users, computer systems and agents, conduct 
powerful. While early computer programs were limited to negotiations on behalf of their client, and enter into corn- 
performing only basic mathematical calculations, current mercial transactions. 

computer programs handle complex tasks such as voice and 45 The relative performance of intelligent agents in handling 

image recognition, predictive analysis and forecasting, mul- certain tasks may vary between agents, e.g., depending upon 

timedia presentation, and other tasks that are too numerous the degree of domain knowledge possessed thereby, 

to mention. However, an agent which performs well when handling a 

However, one common characteristic of many computer particular task in one instance may not perform as well when 

programs is that the programs are typically limited to 50 handling other tasks, or even when handling the same type 

performing tasks in response to specific commands issued of task in other situations and under other circumstances. As 

by an operator or user. A user therefore must often know the a typical agent may experience a wide variety of situations 

specific controls, commands, etc. required to perform spe- when it is operating in a real world environment, a given 

cific tasks. As computer programs become more complex agent will not always be able to provide its client with 

and feature rich, users are called upon to learn and under- 5S optimal performance. Moreover, it is often difficult, if not 

stand more and more about the programs to take advantage impossible, to develop an agent that is capable of handling 

of the improved functionality. all possible situations with equally proficient results. 

In addition to being more powerful, computers have also For example, in electronic commerce applications, secu- 

become more interconnected through private networks such rity is a major concern. Intelligent agents may be entrusted 

as local area networks and wide area networks, and through 60 with the authority to sell or purchase products on behalf of 

public networks such as the Internet. This enables computers a client, and may be dispatched to any number of computer 

and their users to interact and share information with one systems in a network to perform negotiations. Often, the 

another on a global scale. However, the amount of infor- computer systems to which the agents are dispatched are 

mation is increasing at an exponential rate, which makes it remote to the client's system, and thus not under the direct 

increasingly difficult for users to find specific information. 65 control of the client. An agent may thus be subject to 

As a result of the dramatic increases in the both com- scrutiny by third parties, either through monitoring trans- 

plexity of computer programs and the amount of information missions between the agent and its client, or through actual 
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scanning and reverse compiling of the program code of the negotiations, with the projected security risk for a given 

agent. In some instances, the risk may be segregated into negotiation being used as the objective criteria by which 

transmission risk, which is associated with the transmission agents are selected to conduct the negotiations. In unknown 

media between the agent and the client, and environment or untested environments, or in environments in which 

risk, which is associated with the environment in which the 5 known security risks are present, a limited intermediary 

agent is resident. Each type of risk may present unique agent may bc dispatched, leaving the majority of domain 

concerns for a given agent. knowledge such as authority to complete transactions with 

Beyond the clear risk of discovering confidential infer- thc clicnt For other environments where the security risk is 

mation such as a client's credit card or bank account number, known to be not as great, agents having more autonomy to 

additional risks are posed when an agent is subject to 10 conduct negotiations and complete transactions independent 

scrutiny. As an example, an agent would be placed at a of me client may ^ dispatched, thereby relieving the client 

competitive disadvantage if any of its negotiation strategies, of much of me burden of handling these particular negotia- 

techmques or plans, or any specific instructions from its tions However, it should be appreciated that a vast number 

client, were discovered by other parties. If a message to an 0 f objective criteria for selecting different agents may be 

agent from its client indicates that the agent should offer 15 used i n the alternative, and thus the invention is not limited 

$100, but is authorized to go as low as $90, another party t0 this part icular embodiment or application, 

that intercepts this message immediately knows that a trans- ^ and Qther advant and features> which charac . 

action may be complete^ ^ ^ are set forth ia the claims annexed 

message is not intercepted, if the agent has stored the $90 here(Q an(J formi a fufther hereof Ho fof a 

price as its lowest authorized offer, reverse compilation or 20 ^ unders tanding of the invention, and of the advantages 

scanning ot the agent by another party may lead to similar and objectives attained through 

its use, reference should be 

rcs ' made to the Drawing, and to the accompanying descriptive 

The degree of domain knowledge such as autonomy matterj ^ which is described illustrated embodiments 

granted to an agent by its client, and the remaining domain 0 f ^ c mve ntion. 

knowledge retained by the client, may vary significantly for 2 5 

different agents. In general, however, the more domain BRIEF DESCRIPTION OF THE DRAWING 

knowledge that is imparted to an agent, the greater the FIG. 1 is a flowchart illustrating the operation of an agent 

security risk presented to the agent and its client. manager in selecting one of a plurality of intelligent agents 

Often, a tradeoff must be made between absolutely secure with varied degrees of domain knowledge to perform a 

agent transactions with prohibitively high costs, and less 30 computer task, consistent with the invention, 

secure agent transactions with acceptable costs. FIG. 2 is a flowchart illustrating the operation of an agent 

The greatest security would be obtained when most, if not manager in selecting within an intelligent agent one or more 

all, of the domain knowledge for conducting negotiations is of a plurality of program modules with varied degrees of 

retained by the client, with only a limited intermediary agent domain knowledge to perform a computer task, consistent 

sent out to relay messages between the client and the other 35 with the invention. 

party being negotiated with. However, this configuration FIG. 3 is a flowchart illustrating the operation of an 

would present significant time and cost burdens on the client intelligent agent in selecting one or more of a plurality of 

since all decision making would remain with the client, and program modules therein having varied degrees of domain 

since multiple messages would likely need to be transmitted knowledge to perform a computer task, consistent with the 

between the client and the other party. Given these likely 4Q invention. 

burdens, there would be little, if any, benefit in utilizing an FIG. 4 is a block diagram of a networked computer 

agent at all. This would especially be the case in operating system environment for use with the illustrated embodi- 

environments where the security risks are known to be ments of the invention. 

sma N- FIG. 5 is a block diagram of the networked computer 

Security risks are also present for other applications of 45 S y S t e m of FIG. 4, illustrating the interaction between intel- 

intelligent agents. For example, a client may have developed ligent agents therein. 

an agent with a unique search engine that greatly outper- FIG. 6 is another block diagram of the networked corn- 
forms its competitors' designs. In certain operating p Uter system of FIG. 4, illustrating in greater detail the 
environments, the agent may be more likely to be inter- primary components in the client and remote computer 
cepted and copied by competitors — a situation that the client 50 systems. 

would prefer to avoid. However, without an agent mecha- FIG. 7 is a block diagram of an exemplary agent manager 

msm that appropriately balances cost and security, the and rc i n f orccm ent learning module consistent with the 

industry will continue to be limited by today's solutions. invention, and for use in the selection of one of a plurality 

SUMMARY OF THE INVENTION 5S of agents having varied degrees of domain knowledge. 

The invention addresses these and other problems asso- " FIG - 8 is a flowchart illustrating the operation of a 

ciated with the prior art in providing a manner of optimizing semi-autonomous agent consistent with invention, 

the performance of a given computer task by utilizing a FIG- 9 is a flowchart illustrating the operation of a 

plurality of intelligent agents suited to perform the computer fully-dependent agent consistent with invention, 

task but having varied degrees of domain knowledge. Based 60 FIG - 10 is a flowchart illustrating the operation of a 

upon an objective criteria that is determined for a given fully-autonomous agent consistent with invention, 

situation, one of the plurality of intelligent agents is selected FIG. 11 is a block diagram of an exemplary intelligent 

and dispatched to perform the task. Consequently, the per- agent having multiple program modules of varied degrees of 

formance of the computer task is optimized for a wide domain knowledge. 

variety of situations. 65 FIG. 12 is a block diagram of another exemplary intelli- 

In an exemplary embodiment of the invention, intelligent gent agent constructed from program modules from one or 

agents having varied levels of autonomy are used to conduct more pools of program modules. 
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DETAILED DESCRIPTION OF THE 
ILLUSTRATED EMBODIMENTS 

The embodiments of the invention are principally directed 
to optimizing the performance of computer tasks by intel- 
ligent agents in a wide variety of situations. 

Intelligent agents are computer programs which have 
been delegated a degree of autonomy but which are limited 
to operating within constraints defined by their client (which 
may be, for example, another agent; a computer program, 
application or system; or an individual interacting with the 
agent through a computer — hereinafter a client computer 
process). A subset of such agents which are capable of being 
passed between and operating in different applications or 
computer systems are referred to as mobile agents. In certain 
circumstances, the functions of a mobile agent, combined 
with the functions of an agent manager program that is 
resident in a client's computer system and interacts with the 
mobile agent, may cooperatively be considered as portions 
of a single intelligent agent where the domain knowledge 
has been allocated between home and mobile portions. 
However, as used herein, the term agent may be considered 
to include simply a portion or module of an agent resident 
in a computer system irrespective of whether any portion of 
the functionality for the agent as a whole is resident on 
another computer system or in another computer process. 

In general, an agent has the ability to sense, recognize and 
act. These functions are typically embodied in a number of 
components, including an engine, a knowledge component, 
an adapters component, a library, and a view component. 
Generally, the engine controls the overall operation of an 
agent and functions as the "brains" of the agent, and the 
knowledge component stores information that is represen- 
tative of the acquired knowledge of the agent. The adapters 
component handles interaction between an agent and exter- 
nal objects, thereby functioning as the mechanism through 
which the agent "senses" and interacts with its environment. 
A library persistently stores the information utilized by the 
knowledge component, and the view component provides 
the human interface, if any, for an agent, e.g., for supplying 
instructions to the agent. 

Moreover, as discussed above, agents may have varied 
degrees of domain knowledge — that is, intelligence and/or 
skills applied to a specific field or function, typically applied 
to a particular computer task. This may be exhibited through 
greater or lesser degrees of autonomy or authority, alternate 
processing strategies, different types and/or amounts of 
stored information, different sources of information, differ- 
ent capabilities (e.g., monitoring functionality vs. monitor- 
ing and response functionality), different goals or objective 
functions, different inputs or sensed values to which an agent 
may be responsive, alternate logic implementations (e.g., 
neural networks vs. procedural logic), etc. 

For example, in the illustrated embodiments discussed in 
greater detail below, domain knowledge is represented in 
part by the degree of autonomy granted to an agent to 
conduct negotiations in electronic commerce applications as 
well as the inputs to which the agent is responsive and the 
knowledge present in the agent. In addition, in other elec- 
tronic commerce applications, domain knowledge may rep- 
resent different manners or strategies for achieving a desired 
goal, e.g., optimizing a negotiation strategy for one agent to 
maximize profit, while optimizing a strategy for another 
agent to maximize the number of completed transactions. 
Domain knowledge in such applications may also be market 
dependent, e.g., where agents are optimized for automobile 
markets, stock markets, etc. 
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As another example, domain knowledge may represent a 
particular indexing or searching algorithm used by a data 
mining agent, e.g., where one particular searching algorithm 
may work better for legal cases while another may work 

5 better for technical documents. In other data mining 
applications, domain knowledge for one type of agent may 
include only the ability to monitor for problems, e.g., 
looking for low stock or late deliveries in an inventory 
monitoring agent. In other agents, the domain knowledge 

10 may also include the ability to respond, e.g., in the above 
inventory application, ordering low stock items, ordering 
alternates for late deliveries, or canceling orders when sales 
are low. 

Domain knowledge may also simply represent different 

15 manners of performing the same or similar general task, e.g., 
for the purpose of disguising the identity of the agents' 
client, since differently-configured agents may present dif- 
ferent characteristics when scanned by third parties. For 
example, this may permit differently-configured agents to be 

20 dispatched from a single client to conduct negotiations with 
a single party so that the agents play off one another to obtain 
more favorable results for the client. 

Intelligent agents are typically programmed to handle 
particular computer tasks, which may include practically 

25 any operation performed on a computer. For example, in 
electronic commerce applications, the computer tasks for 
which agents are programmed to handle are typically that of 
conducting negotiations and completing transactions. For 
data mining applications, the computer tasks may be the 

30 indexing or retrieval of desired information from one or 
more databases. 

Consistent with the invention, one or more objective 
criteria are used in the selection of one of a plurality of 

35 agents, or alternatively, one of a plurality of program mod- 
ules in an agent, such that a given computer task is handled 
with optimal performance for its given situation. The plu- 
rality of agents (or program modules) are all configured to 
handle a common general computer task (e.g., "buy a car", 

4Q "get me everything you can find out about company X", 
etc.), although the relative performance, capabilities, 
manners, and mechanisms by which the different agents (or 
program modules) operate to handle the task may nonethe- 
less differ. 

45 A program module (or mechanism) may be considered to 
include any function, object, or block of program code 
configured to perform a given computer task. A suitable 
objective criteria may represent practically any testable or 
measurable variable or variables that varies in different 

50 situations to which an agent may be exposed, and that may 
affect an agent's relative performance under those situations. 

For example, in the illustrated embodiments discussed in 
greater detail below, the objective criteria is the perceived 
risk to an agent dispatched to a remote computer system, 

ss whether due to known risks or due to a lack of knowledge 
as to the risk inherent in an untested environment. The 
objective criteria may also include relative compute or 
transmission speeds, or the perceived risk of a particular 
transmission media, which may be used to determine 

6Q whether domain knowledge related to disguising or encrypt- 
ing messages is implemented in a dispatched agent. 

Other examples of objective criteria may include the 
source of information for data mining applications (e.g., 
particular domains or databases, or types of data), user 

65 approval or feedback (e.g., a qualitative assessment of 
performance by a user), and profits or costs for electronic 
commerce applications, among others. 
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Selection of agents or program modules within agents Similarly, the program flow for selecting one or more of 

based upon the objective criteria will vary depending upon a plurality of program modules based upon an objective 

the particular objective criteria and types of situations to criteria is illustrated by agent manager 210 in FIG. 2. In 

which the agent(s) may be exposed. For example, for the block 211, a request to perform a task is received, and in 

illustrated embodiments discussed in greater detail below, 5 block 212 the objective criteria is tested by the agent 

the risk to an agent may be tested based upon the IP address manager. 

of the remote system to which an agent is to be dispatched, Next, in block 214, one or more program modules are 

and any identifying information about the other party to a selected, based upon the objective criteria, from a plurality 

negotiation, among others. Separate risk factors may also be 0 f program modules having varied degrees of domain 

monitored for the transmission media and the agent's resi- JQ knowledge. In some embodiments, one of a plurality of 

dent environment such that domain knowledge specific to available program modules may be selected. For example, 

each type of risk may be separately utilized as necessary. program modules may have duplicate functionality that is 

For data mining applications, the source of information implemented in alternate manners. In such instances, only 

could be tested based upon the IP address of the database one module may be selected. 

being searched, the type of information being sought, etc. In J5 In other embodiments, individual program modules may 
electronic commerce and other applications, the reliability have additive or non-overlapping functionality so that mul- 
of data could also be tested by looking at the source thereof tiple program modules (e.g., a subset of program modules) 
(e.g., prices retrieved from a bulletin board may be more arc selected in a given agent. For example, one program 
reliable than prices quoted in a magazine article), and/or by module may provide basic domain knowledge, with addi- 
comparing the similarity of the data to the type of informa- 2Q ti onal p r0 gram modules selected as necessary to add addi- 
tion being sought. Also, the volatility of a market or the type, t i ona i domain knowledge to more sophisticated agents, 
quandtyorvalueofparticularclassesofgoodsmaybetested [n addition> lhe t need tQ be ^fig^ for 
(e.g., a car buying agent may warrant or require a more execution of the appropriate program module or 
aggressive negotiation strategy than a compact disc buying modules Fof example> if an agent is provided ^ bom 

agent;. 25 selected and non-selected program modules (e.g., where 

In addition, selection based upon an objective criteria may duplicative functionality is provided by different alternative 

occur dynamically, and may even be updated from time to mo dules), an agent may be configured by initializing, send- 

time. For example, an agent may be configured to dynami- mg a mesS age or setting a variable in the agent specifying 

cally select program modules in response to changing cir- which pr0 gram module or modules to execute. In the 

cumstances such as changes in market conditions, changes 30 alternative, an agent may be constructed from component 

in size, speed, or other capacity of a remote site, and new parts pr i or t0 ^ dispatch, whereby the agent may be 

input from a client (e.g., as related to information that is configured by bundling all of the selected program modules 

otherwise unavailable to an agent while resident in a remote ^th any necessary components that are generic to all of an 

system), among others. agents' permutations. 

Agents or program modules may be matched with par- 35 N extj m block 216, the configured agent is dispatched to 

ticular values of an objective criteria to assist in the selection perform its task, and in block 218, the performance of the 

of agents or program modules. The matching may be as agent upon comp letion of the task is optionally analyzed if 

simple as a mapping or table matching agents/program adaptive selection is utilized. 

modules and values of the objective criteria, or may include u should also bc appreciated that me functions of testing 

predictive and interpretive logic such as provided by factor 40 the objective criteriaj selecting agents or agent program 

weighting, neural networks, expert systems, fuzzy logic mo dules based upon the obi ec ;tive criteria, and/or analyzing 

systems, etc. to attempt to select the most appropriate lhe performanC e G f agents need not be performed by an 

agent/program module even when no direct match exists for agent manager Qf Qther ^puter process resideat in the 

a given value of the objective criteria. clieat mmpa ^ r system, but may be performed by any 

Moreover, selection based upon the objective criteria may 45 suitable evaluation module or program. In addition, perfor- 

be adaptively controlled. For example, the actual results and m ance anaiyzation may also rely on a user's subjective 

relative performance of agents dispatched to perform par- assessment. A separate agent may be utilized as an evalua- 

ticular tasks may be analyzed and used to modify the tion module t0 perform any or all of mcsc f unct i 0 ns. Also, 

selection of agents for given values of the objective catena. any of lhese felons may be implemented in the agent that 

To this extent, a reinforcement learning module, imple- 50 has been dispatched to perform a task, whereby the agent 

mented for example in a neural network, may be particularly includes an evaluation module. 

well suited for use in agent or program module selection For example> „ ^touted in FIG. 3, an agent 220 may 

consistent with the invention. receive a Usk request in block 221> test lhe objective criteria 

As discussed above, the objective criteria may be used to in block 222, select the appropriate program module in block 

select between a plurality of agents, or between a plurality 55 2 24 and execute the selected program module in block 226 

of program modules in an agent, that have varied degrees of t0 per f 0 rm the task. In addition, the agent may analyze its 

domain knowledge. The program flow for dispatching one of performance in block 228 if desired. The above-described 

a plurality of agents based upon an objective criteria is configuration has a number of benefits. For example, in 

illustrated by agent manager 200 in FIG. 1. In block 201, a electronic commerce applications, an agent could be dis- 

requcst to perform a task is received, and in block 202 the 60 patched with a task to seek out negotiations on multiple 

objective criteria is tested by the agent manager. Next, in remote syste ms, whereby the agent would have the ability to 

block 204, an agent is selected, based upon the objective dynamically determine the risk at each remote system and 
criteria, from a pool of agents having varied degrees of aIter its behavior accordingly, 
domain knowledge. In block 206, the selected agent is 

dispatched to perform its task, and in block 208, the per- 65 Exemplary Hardware/Software Environment 

formance of the agent upon completion of the task is A representative hardware environment suitable for use 

optionally analyzed if adaptive agent selection is utilized. with the illustrated embodiments of the invention is illus- 
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trated in FIG. 4, where a networked computer system 10 provided through interface 50. Agent manager program 32 is 

generally includes one or more computer systems, e.g., resident in memory 30, as is a reinforcement learning 

single-user computer systems 16, 18 and multi-user com- module 34 that enables the agent manager to adaptively 

puter systems 20, 60, coupled through a network 15. Multi- optimize the performance of tasks. Storage 40 includes one 

user computer system 20 typically includes one or more 5 or more agents 42 (of which may include agent 100, for 

servers 25 to which one or more single-user computers 22 example), which are computer programs or modules that 

may networked through a separate network 24. Similarly, may be retrieved and used locally within system 20, or 

multi-user computer system 20 typically includes one or dispatched to remote systems to execute and perform tasks 

more servers 65 coupled to one or more single-user com- on behalf of the client system. Storage 40 also includes an 

puter systems 62 through a network 64. Network 15 may 10 agent mission database 44 which may track agent operations 

represent any type of networked interconnection, including and the relative success or failure thereof, 

but not limited to local-area, wide-area, wireless, and public Server 65 of remote system 60 also includes a CPU 68 

networks (e.g., the Internet). coupled to a memory 70, storage 80, external network 

It is anticipated that agents consistent with the invention connection 90 and local network connection 92 over a bus 

may originate in and be resident from time to time on any of 35 94. An agent host program 72 is resident in memory 70 to 

the above-mentioned computer systems. One possible dis- handle interactions between agents resident in the remote 

tinction between the computer systems for the purposes of system. Typically, the agent host program is an asynchro- 

the invention may be whether each is a client or a remote nous message/event driven environment that provides a 

system relative to a particular agent. For example, FIG. 5 common platform over which agent computer programs 

illustrates an embodiment of computer system 10 where 20 execute and interact, much like an operating system. The 

multi-user computer system 20 is a client system, and agent host is also capable of permitting messages to be sent 

multi-user computer system 60 is a remote system. between agents and their clients. Memory 70 also includes 

A client system will hereinafter refer to a computer a negotiating program 74 which operates as the "other party" 

system that provides an agent a certain level of security from in transactions with agent 100, which may be another agent, 

manipulation by other parties when the agent is resident on 25 a market or bulletin board application, or even an interface 

the system. The client system is also the computer system program through which an individual interacts with agent 

from which management of the agent is typically handled. 1W. Storage 80 maintains a transaction history database 82 

The agent typically but not necessarily will also originate which logs the transactions completed on the server, 

from the client system. Servers 25, 65 may be, for example, AS/400 midrange 

A remote system, on the other hand, will hereinafter refer 30 computers from International Business Machines Corpora- 

to a computer system that is typically not capable of pro- tion. However, it should be appreciated that the hardware 

viding a desired level of security for an agent, generally embodiments described herein are merely exemplary, and 

because the computer system is not under the control of the that a multitude of other hardware platforms and configu- 

client. It is typically while resident on a remote system that 35 rations may be used in the alternative, 

an agent runs the greatest risk of being scanned or reverse Moreover, while the invention has and hereinafter will be 

compiled, or of having communications intercepted or described in the context of fully functioning computer 

monitored, by other parties. systems, those skilled in the art will appreciate that the 

The various embodiments described hereinafter have various embodiments of the invention are capable of being 

principal uses in electronic commerce applications, where 40 distributed as a program product in a variety of forms, and 

agents are configured to negotiate commercial transactions, that the invention applies equally regardless of the particular 

generally in the role of buying or selling agents. The agents type of signal bearing media used to actually carry out the 

may negotiate with other agents, other computer systems, or distribution. Examples of signal bearing media include but 

even other individuals. The agents may interact one-on-one, are not limited to recordable type media such as floppy 

ormaybecapableofoperatingwithina"market ,, ofmultiple 45 disks, hard disk drives, and CD-ROM's, and transmission 

agents, along the lines of a stock or commodity market. type media such as digital and analog communications links. 

Computer systems having the ability to host agents for XT . . * . o i 

interaction therebetween include negotiating programs of Negotiating Agents with Adaptive Agent Selection 

varying sophistication and are hereinafter referred to as FIG. 7 illustrates an exemplary configuration of client 

agent hosts. However, it should be appreciated that the 50 computer system 20 for providing adaptive agent selection 

invention applies equally to other applications of intelligent consistent with the invention. Agent manager 32 is coupled 

agents, and thus should not be limited specifically to elec- to a reinforcement learning module 34 through which analy- 

tronic commerce applications. sis of agent performance and control of agent selection is 

FIG. 5 illustrates a mobile intelligent agent 100 which performed. Agent manager 32 is shown coupled to a pool of 

communicates with an agent manager 32 in client system 20. 55 agents 42 having varied degrees of domain knowledge, or 

During negotiation with another party such as negotiating alternatively, to one or more pools of program modules 42' 

agent 95, mobile agent 100 is resident on remote system 60. having varied degrees of domain knowledge. A dispatched 

It should be appreciated that remote system 60 may be the agent 100 communicates with agent manager 32 to conduct 

client for agent 95, or may also be considered to be remote negotiations on behalf of the client, 

relative to this agent as well. $q One or more information sources 56 are also coupled to 

An exemplary functional design of networked computer agent manager 32 to enable it to obtain the factor or factors 

system 10 for implementing various embodiments of the for testing the objective criteria (in this embodiment, secu- 

invention is illustrated in FIG. 6. Server 25 of client system rity or risk). Information sources 56 may include internal 

20 generally includes a central processing unit (CPU) 28 databases updated by manager 32 or other components in 

coupled to a memory 30 and storage 40 over a bus 54. A 65 client system 20, as well as external databases, 

local area network interface is provided at 52, and an For example, risk may be determined by maintaining a 

interface to remote system 60 over external network 15 is database of one or more risk factors for particular 
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destinations, addresses or parties being negotiated with. To and generates a scalar reinforcement reward/penalty value 

this extent, information about a remote system or other therefrom. A critic network 37 provides temporal difference 

agents may be obtained by the agent manager, such as a learning in response to the actual agent performance data 

name or identification, an operating system, a type of and the reinforcement value from the sensor. Critic network 

communication port, a client, a bank and/or bank account s 37 outputs a utility signal that is back propagated to a 

number, a homebase address, the name or identification of plurality of action networks 38. 

an agent program, the size of an agent program, where ^ act j on network 38 is provided for each agent or 

messages and other communications with an agent originate, program module. Each action network receives agent per- 

and/or the pattern of input/output (I/O) compared to CPU formance data and outputs therefrom an action merit signal 

cycles for I/O transmissions. Also, the agent manager may 10 l0 a stochastic action behavior block 39. Block 39 probabi- 

attempt to retrieve a credit card number or bank account fistically chooses an action (here, which one of the agents or 

number from an unknown agent and validate the number. program modules to dispatch), and outputs therefrom a 

Moreover, an unknown agent may be scanned and compared control action signal representative thereof to agent manager 

to other known agents, e.g., comparing the percentage of 32. Block 39 introduces randomness to agent selection to 

identical code, determining the language the agent was 15 widen its search space, which has been found to improve 

written in, or searching for unique patterns in much the same j ong term performance of the module, 

manner as a virus checking program. M(xJule 34 adaptivcly mod ifics its selection of agents 

Risk may also be analyzed over time based upon the based upon the analyzed performance of a dispatched agent, 

actual performance of dispatched agents, e.g., by monitoring jf an agcnt performs well, sensor 36 reinforces critic net- 

the number of dispatched agents that disappear, or the length 20 wor k 37, which in turn reinforces the corresponding action 

of time between communications from dispatched agents. network 38 such that it is more likely that the same agent/ 

Also, risk may be analyzed by monitoring dispatched agents program module is selected under similar circumstances. On 

for tampering, e.g., by performing checksums or cyclic the other hand, if an agent performs poorly, sensor 36 

redundancy checks, analyzing binary signatures, etc. Either negatively reinforces critic network 37, which in turn 

the agent manager or the agents themselves may be used to 25 rec j uces ^ utility output to action networks 38 to reduce the 

detect any tampering. Risk may also be analyzed by moni- likelihood that the same agent/program modules is selected 

toring the behavior of other parties being negotiated with. under similar circumstances. 

In addition, risk may be measured based upon the prof- I Q the alternative, other reinforcement learning modules 

itability of agents. For example, the performance of a ^ an d algorithms may be used. For example, other neural 

negotiating agent may be measured in terms of the number network architectures, such as Q -Learning neural networks, 

of complete transactions, gross income of all transactions, Bayesian networks, dynamic programming techniques, 

net profit of all transactions, etc. The performance may then among others, may be used. 

be offset by the costs of using the agent, e.g., in terms of the Module 34 may be initially trained in a manner known in 

time required to complete transactions, local/remote CPU tne art> e g ( by using examp i e training records. In the 

charges, communication (on-line) charges, etc. If an agent is alternative, module 34 may simply be initialized with pre- 

compromised in some way, where another party gains a determined or random networks which act as a starting 

negotiating advantage, a negative impact on profitability reference point from which the module adaptively learns, 

would be observed, The former approach often improves the quality of the initial 

Agent manager 32 provides inputs to reinforcement learn- 4Q decisions made by the module in operation, 

ing module 34 to permit module 34 to perform the functions j^ta pool 42 includes a plurality of agents in which the 

of estimating risk over time and selecting agents or agent domain knowledge thereof varies in the degree of autonomy 

program modules. Module 34 maintains a mapping between delegated thereto during negotiations, the internal knowl- 

agents/program modules and different levels or types of risk, cdge therein, and the inputs to which an agent is responsive 

so that an appropriate agent/program module is selected 45 thereto. At one extreme, a simple intermediary agent, or 

depending upon the measured level or type of risk to a ^ dependent agent, may have little domain knowledge in 

dispatched agent under the current circumstances. t hat the agent merely serves the function of passing mes- 

Agent manager 32 provides optimum agent performance sages between a client and another party to a negotiation, 

data related to a desired task for which to test the objective such that authority for determining offers, accepting or 

criteria. In response, module 34 outputs a control action 50 rejecting offers, implementing negotiation strategies, etc. 

output back to agent manager 32 to control which agent or rests fully with the client. At the other extreme, a fully 

agent program module should be selected and dispatched for autonomous agent may be provided with the basic param- 

performing the given task. Also, when a dispatched agent is e ters for a desired transaction and thereafter entrusted with 

returned and analyzed, agent manager 32 provides actual the authority to complete such a transaction free from any 

performance data to adaptively modify the selection func- 55 additional input on the part of the client. Between these 

tionality of module 34. extremes, one or more additional agents, referred to as 

In the illustrated embodiment, an adaptive heuristic critic semi-autonomous agents, may be entrusted with the ability 
neural network architecture is utilized in module 34 to to conduct limited negotiations. One possible semi- 
provide adaptive reinforcement learning. The design, train- autonomous agent may be authorized to conduct negotia- 
ing and operation of an adaptive heuristic critic neural 60 lions f° r a limited number of offer iterations, but with final 
network is well known in the art. For example, one suitable authority for approving an agreed-upon transaction still 
development environment for performing the development resting with the client. Other possible degrees of domain 
and training of such a neural network is the IBM Neural knowledge between the extremes are also possible. 
Network Utility available from International Business FIG. 8 illustrates an exemplary program flow for nego- 
Machines Corporation, among others. 6 5 tiation with a semi-autonomous agent which has authority to 

Module 34 includes a sensor 36 which compares desired negotiate with another party with limits on the offer amount 

(optimum) agent performance and actual agent performance and the number of iterations (offers made by the agent) that 
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the agent can undertake on its own. The interaction between 
the agent and an agent manager on a client system, and the 
other party being negotiated with (another agent, computer 
process or system, or individual interacting through a 
computer), is also illustrated. 

For the semi-autonomous agent, negotiations are insti- 
tuted after the agent manager selects the agent from the 
agent pool and dispatches the agent, as illustrated in blocks 
130 and 131. It should be appreciated that the agent may be 
dispatched with a desired transaction or transactions which 
the client wishes to complete (including information such as 
desired quantity, description of goods/services to be sold/ 
purchased, etc.), as well as any other constraints placed upon 
the agent by the manager (e.g., the number of negotiation 
iterations to process, a maximum or minimum limit on the 
amount to offer, etc.). In the alternative, some or all of this 
information may be relayed to the agent at a later time by the 
agent manager. 

Next, in block 132, the agent gathers information after it 
has been dispatched to the remote system. For example, the 
agent may poll a bulletin board or market for outstanding 
offers, or may attempt to Gnd and initiate negotiations with 
another party, e.g., another computer system, another agent, 
or an individual interacting through a computer system. The 
agent may also simply wait for incoming offers from other 
parties. Block 132 also performs the functions of processing 
messages from other parties and from the agent manager. 

Next, in response to input received by the agent, a 
decision is made in block 134, It is in this block that a 
negotiation strategy of the agent is implemented, whereby 
the agent determines, based upon the input it receives, 
whether and how to proceed in negotiations with another 
party. 

Generally, one of three actions may be instituted in 
response to block 134. First, as illustrated by block 136, a 
message may be sent to the other party, e.g., indicating 
acceptance or rejection of an outstanding offer, issuing an 
offer or counteroffer, terminating negotiations, requesting 
that the other party wait, etc. The other party receives the 
message at block 138 and processes it at block 140, typically 
issuing a response message back to the agent. Control then 
returns to block 132 to process the response message from 
the other party. 

Second, as illustrated by block 142, the agent may decide 
to send a message back to the agent manager. The agent 
manager then receives the message at block 144, processes 
the message in block 146 and sends a response message back 
to the agent in block 148. In block 150, the agent receives 
the message and passes control to block 132 to process the 
response message from the agent manager. Also, as dis- 
closed in U.S. patent application Ser. No. 08/822,119, 
entitled "APPARATUS AND METHOD FOR COMMUNI- 
CATING BETWEEN AN INTELLIGENT AGENT AND 
CLIENT COMPUTER PROCESS USING DISGUISED 
MESSAGES" (which has been incorporated by reference), 
the messages between the agent and its manager may be 
disguised to impede scrutiny by third parties. 

Third, as illustrated by block 152, the agent may decide to 
leave the remote system and return to its client system, e.g., 
after completion of a successful transaction or after termi- 
nation of an unsuccessful negotiation. An agent may "leave" 
when its program code is moved or transmitted from the 
remote system to the client system. In the alternative, an 
agent may transmit any necessary information (e.g., domain 
knowledge and/or results) to the client system and terminate 
on the remote system. 
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The return of the agent is illustrated in block 154. Then, 
in block 156, the performance of the agent may be analyzed 
in the manner discussed above. 

FIG. 9 illustrates an exemplary program flow for nego- 

5 tiation with a fully-dependent agent which acts principally 
as an intermediary with no authority to negotiate with 
another party without specific instructions from the agent 
manager. As with the semi-autonomous agent, negotiations 
are instituted after the agent manager selects the agent from 

io the agent pool and dispatches the agent, as illustrated in 
blocks 160 and 162. Next, in block 164, the agent gathers 
information after it has been dispatched to the remote 
system. Block 164 also performs the functions of processing 
messages from other parties. 

15 Next, in response to input received by the agent (typically 
as a result of an offer being received from another party), a 
message is sent to the agent manager in block 166. The agent 
manager then receives the message at block 168, processes 
the message in block 170 and sends a response message back 

20 to the agent in block 172. As with the semi-autonomous 
agent, messages between the agent and agent manager may 
be disguised to impede scrutiny by third parties. 

Next, in block 174, the agent receives the message and 
passes control to block 176 to process the instruction pro- 

25 vided in the response message. Generally, one of two actions 
may be instituted in response to block 176. First, as illus- 
trated by block 178, a message may be sent to the other 
party, e.g., indicating acceptance or rejection of an outstand- 
ing offer, issuing an offer or counteroffer, terminating 
negotiations, requesting that the other party wait, etc. The 
other party receives the message at block 180 and processes 
it at block 182, typically issuing a response message back to 
the agent. Control then returns to block 164 to process the 

35 response message from the other party. 

Second, as illustrated by block 184, the agent may decide 
to leave the remote system and return to its client system, 
e.g., after completion of a successful transaction or after 
termination of an unsuccessful negotiation. The return of the 

40 agent is illustrated in block 186. Then, in block 188, the 
performance of the agent may be analyzed in the manner 
discussed above. 

FIG. 10 illustrates an exemplary program flow for nego- 
tiation with a fully- autonomous agent which has full author- 

45 ity to conduct negotiations with little or no supervision by 
the agent manager. As with the semi- autonomous agent, 
negotiations are instituted after the agent manager selects the 
agent from the agent pool and dispatches the agent, as 
illustrated in blocks 190 and 191. Next, in block 192, the 

50 agent gathers information after it has been dispatched to the 
remote system. Block 192 also performs the functions of 
processing messages from other parties. 

Next, in response to input received by the agent (typically 
as a result of an offer being received from another party), 

55 control passes to block 193 to make a decision based upon 
the inputs to the agent. It is in this block that a negotiation 
strategy of the agent is implemented, whereby the agent 
determines, based upon the input it receives, whether and 
how to proceed in negotiations with another party. 

60 Generally, one of two actions may be instituted in 
response to block 193. First, as illustrated by block 194, a 
message may be sent to the other party, e.g., indicating 
acceptance or rejection of an outstanding offer, issuing an 
offer or counteroffer, terminating negotiations, requesting 

65 that the other party wait, etc. The other party receives the 
message at block 195 and processes it at block 196, typically 
issuing a response message back to the agent. Control then 
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returns to block 192 to process the response message from 
the other party. 

Second, as illustrated by block 197, the agent may decide 
to leave the remote system and return to its client system, 
e.g., after completion of a successful transaction or after 
termination of an unsuccessful negotiation. The return of the 
agent is illustrated in block 198. Then, in block 199, the 
performance of the agent may be analyzed in the manner 
discussed above. 

Any number of negotiation strategies and features may be 
incorporated into any of the agents in agent pool 42. For 
example, U.S. patent application Ser. No. 08/821, 
935entitled "INTELLIGENT AGENT WITH NEGOTIA- 
TION CAPABILITY 30 AND METHOD OF NEGOTIA- 
TION THEREWITH", which has been incorporated by 
reference, discloses a suitable negotiation strategy which 
relies on the previous and current offers from the agent as 
well as the previous and current prices offered by the other 
party. A number of additional negotiation features imple- 
mented in the aforementioned application may also be used, 
including dynamic value determination, disguising negotia- 
tion strategy by randomizing an agent's behavior, and lim- 
iting unproductive negotiations by constraining an agent's 
behavior, among others. Other negotiation strategies and 
features may also be used consistent with the invention. 

Moreover, any of the agents in agent pool 42 may be 
implemented using neural networks for implementing deci- 
sion making and/or message disguising, as disclosed in U.S. 
patent application Ser. No. 08/822,119, entitled "APPARA- 
TUS AND METHOD FOR COMMUNICATING 
BETWEEN AN INTELLIGENT AGENT AND CLIENT 
COMPUTER PROCESS USING DISGUISED MES- 
SAGES" (which has been incorporated by reference). For 
example, for a semi-autonomous agent, a neural network 
may be configured to receive the following inputs: 

other party's last price 

other party's current price 

agent's last public offer 

product characteristics 

instruction from manager: 
increase in offer authorized 
increase in iterations authorized 
increase in offer not authorized 
increase in iterations not authorized 

last message sent back to manager 

In addition, the neural network may be configured to 
generate the following outputs: 

public price offer 

message to other party 
make offer 
withdraw offer 
accept 
reject 

counteroffer 
please wait 
finished 

message to manager 
probable acceptable alternative, do you accept 
iterations exceeded, recommend continue 
iterations exceeded, recommend withdrawal 
recommend immediate withdrawal 
dummy transmission to motivate negotiation 
request approval to increase offer 

For a fully-dependent agent, a neural network may be 
configured to receive the following inputs: 
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30 



35 



45 



50 



55 



60 



65 



other party's last price 
other party's current price 
number of iterations 
agent's last public offer 
product characteristics 
instruction from manager: 

buy or sell 

withdraw 

counteroffer 

In addition, the neural network may be configured to 
generate the following outputs: 
public price offer 
message to other party 

make offer 

withdraw offer 

accept 

reject 

counteroffer 

finished 
message to manager 

current price offer 

other party's current price 

feature presence 
For a fully-autonomous agent, a neural network may be 
configured to receive the following inputs: 
other party's last price 
other party's current price 
number of iterations 
agent's last public offer 
product characteristics 

In addition, the neural network may be configured to 
generate the following outputs: 
public price offer 
message to other party 

make offer 

withdraw offer 

accept 

reject 

counteroffer 
finished 

Moreover, for training purposes, additional information 
may also be used in the generation of suitable training 
records for any of the above agents, although such infor- 
mation is not directly input to the neural network. Such 
information may include: 

desired price and features 

agent autonomy price range (range of prices within which 
agent is authorized to negotiate autonomously) 

manager approval price range (range of prices within 
which agent requires approval from manager) 

acceptable iterations allowed without approval 

It should be appreciated that other agents implementing 
alternate negotiation strategies and decision making 
processes, and with other degrees of autonomy between 
fully-autonomous and fully-dependent, may be used in the 
alternative. Moreover, it should be appreciated that other 
functionality may be required for the autonomous operation 
of any of the agents in agent pool 42, and that such 
additional functionality may be implemented via procedural 
logic and/or neural networks. For example, functions such as 
initialization, maintenance, finding other agents or markets 
to interact with, etc. may also be utilized. However, as these 
functions relate more to the basic operation of an agent, 
which is in general known in the art, these functions will not 
be discussed in any greater detail herein. 
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Referring again to FIG. 7, as discussed above, agent modules to enhance the domain knowledge of the dispatched 

manager 32 may select from multiple program modules of agent. For example, in one embodiment, a pool of optional 

varied degrees of domain knowledge to perform a desired modules 324 may be provided with, among other modules, 

computer task. Agent manager 32 may utilize an agent from a time monitor module 319 that permits the negotiation 

pool 42 having multiple program modules with varied 5 strategy of the agent to vary depending upon the time of day 

degrees of domain knowledge, and then configure the agent (e.g., for markets which tend to be more volatile at the end 

to execute a selected program module based upon the of the day, module 319 may implement a more conservative 

objective criteria. negotiation strategy than otherwise employed). Other 

For example, as shown in FIG. 11, agent manager 32 may optional functionality may be implemented. For example, 

utilize an agent 300 that includes a main module 302 which 10 agent performance monitoring may optionally be included 

oversees the operations of the agent, and a communications within agent 310 if desired. 

module 304 that handles communications between the agent Any number of programming languages and underlying 

and its manager, and between the agent and other parties. platforms may be utilized to implement agents consistent 

Moreover, one or more additional program modules with the invention. For example, Java and other object 

306-308 are interconnected to main module 302 to provide 15 oriented programming languages are particularly well suited 

alternate functionality for the agent. Program module 306 for use with agents implementing multiple program modules 

implements a semi-autonomous functionality which oper- having varied degrees of domain knowledge, as separate 

ates in the manner disclosed above for the semi-autonomous classes implementing each of the alternate program modules 

agent of FIG. 8. Program module 307 implements a fully- may be interconnected with one another in a single package, 

dependent functionality which operates in the manner dis- 20 Other modifications may be made to the illustrated 

closed above for the fully -dependent agent of FIG. 9. embodiments without departing from the spirit and scope of 

Program module 308 implements a fully-autonomous tunc- the invention. Therefore, the invention lies solely in the 

tionality which operates in the manner disclosed above for claims hereinafter appended, 

the fully-autonomous agent of FIG. 10. What is claimed is: 

Selection of which of modules 306-308 may be made by 25 1. A method of handling a computer task using an 

agent manager, in the manner disclosed above with refer- intelligent agent, the method comprising the steps of: 

ence to FIG. 7. In the alternative, selection of program (a) based upon an objective criteria, selecting for dispatch 

modules based upon an objective criteria may be made a selected intelligent agent from a plurality of intelli- 

internally within the agent, e.g., using predetermined selec- ge nt agents having varied degrees of domain 

tion criteria (e.g., to employ different negotiation strategies 30 knowledge, wherein the plurality of intelligent agents 

depending upon the time of day, etc.), or using an optional are configured to handle a common computer task, 

reinforcement learning module 309 which operates in a wherein a non-selected intelligent agent is retained 

similar manner to module 34 discussed above. It should be after selection of the selected intelligent agent; and 

appreciated that when selection is performed within an ( b ) dispatching the selected intelligent agent to handle the 

agent, selections may be updated from time to time to adapt 35 computer task 

the behavior of the agent in view of changing circumstances. 2 method of claim x w h er ein the selecting step 

As also illustrated in FIG. 7, agent manager 32 may rely includes the step of adaptively selecting the selected intel- 

on one or more pools of program modules 42' to construct ^stA agent using a re i n forcement learning algorithm, 

a task-specific agent depending upon an objective criteria. 3 ^ metno d of claim 2, further comprising the steps of: 

For example, as shown in FIG. 12 agent manager 32 may 40 } ob performance information relating to the 

construct an agent 310 from a number of components. First, performance of the selected intelligent agent in nan- 

components which are genenc to all Pf ^ons of an ^ ^ ter lask and 

agent may be utilized, e.g., a main module 312 and com- yt v , . 

munications module 314. ( b ) ^PP^S me performance information to the rein- 
Second, components may be selected from pluralities of 45 A foment Earning algorithm 
alternate program modules. For example, in one embodi- , 4 J** ™ th f of clai l m 2 > w J herein th f reinforcement 
ment where the decision logic of an agent is implemented in learam 8 al g™thm * implemented in an adaptive heuristic 
a neural network, a pool 320 of decision logic neural critic neural network. • • » t . 
network modules may be provided, with another pool 322 of . 5 - J** . method ° f f claim wherein lhe sclecUn S sle P 
price range filters used to optimize the operation of the 50 1Dcludes the ste P s ot: 

neural network modules for different price ranges. The use ( a ) matching each of the plurality of intelligent agents 

of the latter pool 322 permits the modules in pool 320 to be Wlth a valuc of ^ objective criteria; 

price independent so that, when agent manager 32 is (b) determining a selected value of the objective criteria; 

required to dispatch an agent to conduct transactions for a and 

particular product in a particular price range, the agent 55 (c) selecting as the selected intelligent agent an intelligent 

manager merely selects an appropriate decision logic net- agent matching the selected value of the objective 

work module from pool 320 (e.g., semi-autonomous pro- criteria, 

gram module 316) and price range filter from pool 322 (e.g., 6. The method of claim 5, wherein the selecting step 

price range C filter 318). Other manners of customizing the further includes the step of retrieving information for a 

modules in pool 320 may be envisioned, e.g., selecting from 60 selected computer task, wherein the determining step deter- 

different requirements filters to optimize an agent for dif- mines the selected value of the objective criteria using the 

ferent products. Also, both procedural and neural network retrieved information. 

implemented decision logic modules may be included in 7. The method of claim 1, wherein each of the plurality of 

pool 320 so that, depending upon the risk to a dispatched intelligent agents is configured to conduct negotiations in an 

agent, either type of logic may be utilized. 65 electronic commerce application, and wherein the domain 

Third, instead of selecting among alternate program knowledge for each of the plurality of intelligent agents is 

modules, agent manager 32 may add additive program related to the autonomy delegated thereto. 
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8. The method of claim 7, wherein the plurality of 
intelligent agents includes a semi-autonomous agent, a fully- 
autonomous agent, and a fully -dependent agent. 

9. The method of claim 7, wherein the objective criteria 
includes a risk that a dispatched agent is subjected to in 5 
negotiations. 

10. An apparatus for handling a computer task, compris- 
ing: 

a plurality of intelligent agents having varied degrees of 
domain knowledge, wherein the plurality of intelligent 10 
agents are configured to handle a common computer 
task, wherein, based upon an objective criteria, a 
selected agent from the plurality of intelligent agents is 
selected and dispatched to handle the computer task, 
and wherein a non-selected intelligent agent is retained 15 
after selection of the selected intelligent agent. 

11. The apparatus of claim 10, further comprising an agent 
manager configured to dispatch the selected intelligent agent 
based upon the objective criteria. 

12. The apparatus of claim 11, further comprising a 20 
reinforcement learning module, coupled to the agent man- 
ager and configured to adaptively select intelligent agents 
based upon the performance of the plurality of intelligent 
agents in handling the computer task. 

13. The apparatus of claim 12, wherein the reinforcement 25 
learning module comprises an adaptive heuristic critic neu- 
ral network. 

14. The apparatus of claim 11, wherein the agent manager 
is configured to retrieve information for a selected computer 
task, determine a selected value for the objective criteria for 30 
the selected computer task, and select as the selected intel- 
ligent agent one of the plurality of intelligent agents which 

is matched with the selected value of the objective criteria. 

15. The apparatus of claim 10, wherein each of the 
plurality of intelligent agents is configured to conduct nego- 35 
tiations in an electronic commerce application, and wherein 
the domain knowledge for each of the plurality of intelligent 
agents is related to the autonomy delegated thereto. 

16. The apparatus of claim 15, wherein the plurality of 
intelligent agents includes a semi-autonomous agent, a fully- 40 
autonomous agent, and a fully-dependent agent. 

17. The apparatus of claim 15, wherein the objective 
criteria includes a risk that a dispatched agent is subjected to 
in negotiations. 

18. A program product comprising: 45 

(a) a program configured to handle a computer task using 
an intelligent agent, the program comprising a plurality 
of intelligent agents having varied degrees of domain 
knowledge, wherein the plurality of intelligent agents 
are configured to handle a common computer task, 50 
wherein, based upon an objective criteria, a selected 
agent from the plurality of intelligent agents is selected 
and dispatched to handle the computer task, and 
wherein a non-selected intelligent agent is retained 
after selection of the selected intelligent agent; and ss 

(b) a signal bearing media bearing the program. 

19. The program product of claim 18, wherein the signal 
bearing media is transmission type media. 

20. The program product of claim 18, wherein the signal 6Q 
bearing media is recordable media. 

21. A method of handling a computer task on a remote 
computer system using an intelligent agent, the method 
comprising the steps of: 

(a) determining a risk for the remote computer system; 65 

(b) based upon the risk for the remote computer system, 
selecting a selected intelligent agent from a plurality of 
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intelligent agents having varied degrees of domain 
knowledge, wherein the plurality of intelligent agents 
are configured to handle a common computer task on 
the remote computer system; and 
(c) dispatching the selected intelligent agent to handle the 
computer task. 

22. The method of claim 21, further comprising the step 
of matching each of the plurality of intelligent agents with 
a risk level 

23. The method of claim 22, wherein the matching step 
includes the step of adaptively matching each intelligent 
agent based upon the actual performance of the plurality of 
intelligent agents. 

24. A method of handling a computer task using an 
intelligent agent, the method comprising the steps of: 

(a) based upon an objective criteria, adaptively selecting 
a selected intelligent agent from a plurality of intelli- 
gent agents having varied degrees of domain knowl- 
edge using a reinforcement learning algorithm imple- 
mented in an adaptive heuristic critic neural network, 
wherein the plurality of intelligent agents are config- 
ured to handle a common computer task; and 

(b) dispatching the selected intelligent agent to handle the 
computer task. 

25. A method of handling a computer task using an 
intelligent agent, the method comprising the steps of: 

(a) based upon an objective criteria, selecting a selected 
intelligent agent from a plurality of intelligent agents 
having varied degrees of domain knowledge, wherein 
the plurality of intelligent agents are configured to 
handle a common computer task, wherein each of the 
plurality of intelligent agents is configured to conduct 
negotiations in an electronic commerce application, 
and wherein the domain knowledge for each of the 
plurality of intelligent agents is related to the autonomy 
delegated thereto; and 

(b) dispatching the selected intelligent agent to handle the 
computer task. 

26. The method of claim 25, wherein the plurality of 
inteDigent agents includes a semi-autonomous agent, a fully- 
autonomous agent, and a fully-dependent agent. 

27. The method of claim 25, wherein the objective criteria 
includes a risk that a dispatched agent is subjected to in 
negotiations. 

28. An apparatus for handling a computer task, compris- 
ing: 

a plurabty of intelligent agents having varied degrees of 
domain knowledge, wherein the plurality of intelligent 
agents are configured to handle a common computer 
task; and 

a reinforcement learning module configured to adaptively 
select, based upon an objective criteria, a selected 
intelligent agent from the plurality of intelligent agents 
to dispatch to handle the computer task, the reinforce- 
ment learning module including an adaptive heuristic 
critic neural network. 

29. The apparatus of claim 28, further comprising an 
agent manager, coupled to the reinforcement learning mod- 
ule and configured to dispatch the selected intelligent agent. 

30. An apparatus for handling a computer task, compris- 
ing: 

a plurality of intelligent agents having varied degrees of 
domain knowledge, wherein the plurality of intelligent 
agents are configured to handle a common computer 
task, wherein, based upon an objective criteria, a 
selected agent from the plurality of intelligent agents is 



04/23/2004, EAST Version: 1.4.1 



US 6,192,354 Bl 



21 



22 



10 



dispatched to handle the computer task, wherein each 
of the plurality of intelligent agents is configured to 
conduct negotiations in an electronic commerce 
application, and wherein the domain knowledge for 
each of the plurality of intelligent agents is related to 
the autonomy delegated thereto. 

31. The apparatus of claim 30, wherein the plurality of 
intelligent agents includes a semi-autonomous agent, a fully- 
autonomous agent, and a fully-dependent agent. 

32. The apparatus of claim 30, wherein the objective 
criteria includes a risk that a dispatched agent is subjected to 
in negotiations. 

33. An apparatus for handling a computer task on a remote 
computer system, comprising: 

a plurality of intelligent agents having varied degrees of 35 
domain knowledge, wherein the plurality of intelligent 
agents are configured to handle a common computer 
task on a remote computer system, and wherein, based 
upon a risk determined for the remote computer 
system, a selected agent from the plurality of intelligent 20 
agents is dispatched to handle the computer task. 

34. A program product comprising: 

(a) a program configured to handle a computer task using 
an intelligent agent, the program comprising a plurality 
of intelligent agents having varied degrees of domain 
knowledge, wherein the plurality of intelligent agents 
are configured to handle a common computer task, and 
a reinforcement learning module configured to adap- 
tively select, based upon an objective criteria, a 
selected intelligent agent from the plurality of intelli- 
gent agents to dispatch to handle the computer task, the 
reinforcement learning module including an adaptive 
heuristic critic neural network; and 
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(b) a signal bearing media bearing the program. 

35. A program product comprising: 

(a) a program configured to handle a computer task using 
an intelligent agent, the program comprising a plurality 
of intelligent agents having varied degrees of domain 
knowledge, wherein the plurality of intelligent agents 
are configured to handle a common computer task, 
wherein, based upon an objective criteria, a selected 
agent from the plurality of intelligent agents is dis- 
patched to handle the computer task, wherein each of 
the plurality of intelligent agents is configured to con- 
duct negotiations in an electronic commerce 
application, and wherein the domain knowledge for 
each of the plurality of intelligent agents is related to 
the autonomy delegated thereto; and 

(b) a signal bearing media bearing the program. 

36. A program product comprising: 

(a) a program configured to handle a computer task on a 
remote computer system using an intelligent agent, the 
program comprising a plurality of intelligent agents 
having varied degrees of domain knowledge, wherein 
the plurality of intelligent agents are configured to 
handle a common computer task on a remote computer 
system, and wherein, based upon a risk determined for 
the remote computer system, a selected agent from the 
plurality of intelligent agents is dispatched to handle 
the computer task; and 

(b) a signal bearing media bearing the program. 
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